Surveying Haiti's post-quake needs: a quantitative approach

In the aftermath of a disaster, relief workers, community leaders and government authorities must urgently respond to a bewildering constellation of needs, assess disaster-related damage and undertake comprehensive recovery planning, including future risk mitigation. To do this properly it is essential that they acquire a basic understanding of the pre-disaster context, and how things may have changed. In unstable, low-income contexts such as Haiti, administering randomised post-disaster survey research is especially challenging. Census data and public records may be out of date, damaged or inaccessible. Disaster-affected areas may themselves be out of reach.

In situations like these, foreign aid agencies and national governments often resort to ‘quick and dirty’ needs assessments. Because large-scale randomised surveys are considered time-consuming and costly, practitioners often resort to ‘convenience samples’ (readily available groups such as recipients of an existing aid programme) that cannot be generalised to the entire population. Alternatively, they may deploy other methods, such as focus groups, expert surveys or interviews with key informants. As a result, assessments tend to be uneven in quality, narrowly focused and disconnected. A meta-analysis of such studies is often assembled in order to generate an overall picture of a post-disaster situation, despite considerable differences in assessment methodologies. Although such an analysis may provide a valuable historical perspective and in-depth information about particular demographic groups or thematic priorities, it seldom has enough detail to allow for the disaggregation of relationships between variables.

A rigorous quantitative assessment immediately following a natural disaster can generate the requisite data to inform responsive interventions that address the real needs of affected population groups. Moreover, where there is coordination between service providers, government officials and international agencies, such data can be made readily available to relief organisations  and policy-makers. A systematic and representative survey of the population not only allows individuals to inform the policy-making process, but it can also highlight risks and resiliencies not previously identified. Moreover, it can establish a baseline by which the efficacy of post-disaster interventions can be measured, eventually contributing to the growing body of empirical knowledge regarding disaster response. Although such practices are still in their infancy, many NGOs are moving towards results-based and evidence-based approaches to service provision following man-made and natural disasters.

Facing up to challenges

A number of basic obstacles hinder relief agencies and researchers intent on conducting empirical assessments in post-disaster situations. In Haiti, in addition to the sheer physical devastation and wreckage brought about by the recent earthquake, most maps of the capital, Port-au-Prince, were missing streets and even entire neighborhoods. Translators were unavailable or of poor quality; internet access was limited and expensive, and government and international agencies were releasing incomplete and sometimes conflicting reports of damage, mortality and needs.

Although dubbed the ‘Republic of NGOs’, Haiti is home to comparatively few experienced and qualified social science researchers. Many of those living in the country at the time of the earthquake were also personally affected. Homes and offices were lost, along with research data and telephone access, making it difficult to respond to requests to participate in post-disaster assessments. To complicate matters further, the only census data available immediately after the earthquake was collected more than eight years before.

With the world watching on, Haitian government officials and humanitarian agencies faced mounting calls to implement programmes rapidly. They were also expected to deliver results without setting goals or benchmarks of success. Speaking to members of the press in mid-January, one NGO representative complained: ‘We’re not magicians. I can’t conjure up figures out of nowhere. We don’t know how many are dead, buried, or missing … Who knows how many houses have collapsed? What electrical wires are down, what water pipes have burst, or how many people are sleeping in the street’. Even six months on the situation remained confused, with the estimated number of dead ranging from 90,000 to 300,000.

Preparing a post-quake survey

Reliable, valid information was a key priority for policy-makers and relief workers in Haiti. In order to support this effort, the Small Arms Survey, local Haitian researchers, the University of Michigan, the United Nations Development Programme (UNDP) and the International Development Research Centre (IDRC) collaborated on the design and implementation of a peer-reviewed post-disaster assessment. Administered less than two months after the earthquake, the survey was expected to generate baseline data for the World Bank/UN-managed Post-Disaster Needs Assessment (PDNA).

Standardisation

One of the key features of the assessment design was its use of standardised measures to assess specific issues such as physical and mental health, substance use and experiences of violence and crime. The US Department of Agriculture (USDA)’s Food Security Scale-Revised was used to systematically assess levels of hunger and need. Respondents were asked 18 questions about access to food for the entire household, including children and adults.

Where standardised measures were unavailable or inapplicable, an effort was made to incorporate language from previous surveys. For example, information about crime victimisation was obtained using the human rights history framework, a structured interview previously used in Haiti and Lebanon. Demographic information was generated using elements of the International Labour Organisation’s National Child Labor Study, and attitudes to security and security providers were measured using the Gun Attitudes and Ownership Survey, previously applied by members of the research team in both Haiti and the Middle East.

Collaboration

The elaboration of a research plan and household survey instrument began within days of the earthquake. Early planning was undertaken through face-to-face and skype meetings between Haitian social scientists, NGO representatives and leaders and North American research team members. A preliminary plan was prepared and a draft survey circulated and reviewed by all collaborators. Feedback from Haitian community leaders was received during in-person meetings and notes subsequently sent by messenger to the Dominican Republic and then faxed to other research team members, as reliable internet and telephone services were unavailable in Port-au-Prince at the time.

A number of foreign agencies including UNDP and IDRC reviewed the proposal, ensuring that specific variables relevant to the wider recovery effort were included. In order to generate awareness of the impending survey, a summary of findings from earlier household surveys conducted in Port-au-Prince in 2009 was posted to internet-based document-sharing platforms used by agencies involved in the PDNA process. Consequently, additional agencies such as the World Bank contacted the research team to ask that their areas of interest be explored.

A protocol document and questionnaire were finalised, translated into Haitian Creole and submitted to the University of Michigan for ethical approval. The research team then assembled in Haiti to train the enumerators on the new survey instrument. Fieldwork was initiated and results were entered each evening into a simple spreadsheet, so that supervisors could review results and address any data collection errors. From project conception to the release of preliminary findings, the process took a total of 51 days – quite likely the most rapid and inexpensive household survey ever completed in Haiti.

Capacity

Using expatriates in post-disaster situations costs ten to 40 times more than using locals to do the same job. In this case, an experienced team of Haitian enumerators had previously received specialised data collection and research theory training in 2009. New Haitian researchers were mentored and often ‘job-shadowed’ by more experienced team members. Likewise, two Haitian team members attended classes on data analysis software in the latter part of 2009.

The availability of known, enthusiastic and experienced enumerators made it possible to quickly mobilise a post-earthquake survey. Nevertheless, it is important to stress that enumerators need to be well-trained (and possibly retrained) before beginning a large-scale baseline assessment. In 2010, researchers were provided with three training days to enhance data collection and the use of scantrons (paper ‘bubble-entry’ forms that can be read electronically) to record interview responses.

Methods

The 2010 post-quake household survey focused primarily on Port-au-Prince residents, including displaced people residing in settled and ‘spontaneous’ camps. In addition to developing information on the demographic profiles and mortality rate of the population, key themes included victimisation and sexual violence, attitudes towards security providers and perceptions of service delivery, as well as food security, water and sanitation and property issues.

The survey was designed explicitly to compare the experiences and perceptions of security, justice and access to basic services among households, compared with the findings of the 2009 survey. In order to generate a pre- and post- profile, the research team re-interviewed some 1,800 households previously interviewed in 2009, along with an additional 1,147 households residing in 30 camps.

A multi-stage approach was employed to identify households throughout Port-au-Prince and in three over-sampled highly populated zones in both 2009 and 2010. First, a list of random GPS locations was produced, with interviewers identifying all households within a 20m radius at each location before randomly choosing one to interview. Likewise, five large camps were identified using Google Maps, and an estimated population figure for each camp was obtained from the agencies servicing the camp. Camp leaders also provided an estimated population figure and granted permission for enumerators to enter the camp.

Preliminary findings

The survey findings were distributed to PDNA drafters and local authorities in three separate reports in March 2010. A final report was published by UNDP in July 2010 and released in New York and Port-au-Prince.[1] Owing to the varied needs of users, findings were prepared in a variety of formats, ranging from opinion pieces and short executive summaries to longer data-rich reports with tabular annexes. Outputs were circulated through social networking sites, and via UN and NGO networks. While it is not possible to list all of the findings here, a few are featured below.

First, confirming conventional wisdom, children were especially at risk of dying from earthquake-related injuries. They were considerably more likely to have been killed during the earthquake than adults, and were 11 times more likely to have died of injuries after the quake.

Second, while the physical damage was extensive and the requirements of removing rubble and rebuilding are immense, the picture is not necessarily as bad as commonly assumed. Around a third of households suffered no visible damage to their dwellings.

Third, crime rates were dramatically lower than expected. Just 4.1% of all Port-au-Prince households experienced some form of property violation, including theft, vandalism or the intentional destruction of property, in the first two to three months after the earthquake. Unsurprisingly, given the high levels of food insecurity, theft of water and food was most common. Theft was geographically concentrated in certain neighborhoods and usually involved relatively modest amounts.

Fourth, the survey revealed a surprising shift in attitudes towards the Haitian security sector. While reviled by the local population in previous years owing to accusations of systematic human rights violations, the preferred security provider for both the general and camp populations in early 2010 was the Haitian National Police. Likewise, the vast majority of respondents believed that strengthening police capacity would make their communities safer.

Fifth, the survey underlined the importance of social networks – including diaspora links – in influencing how people coped and adapted. For example, the survey observed that almost a quarter (23.4%) of camp households claimed to have received gifts (including remittances) from friends, family, non-governmental organisations and charities in the months following the quake. Remittances are a major source of income for all population groups in Haiti, with some $1.3 billion reportedly flowing into the country in 2009.

Finally, the earthquake response highlighted the critical role played by communications. Well over half of camp respondents said that they received national news from radio. Almost half of the general population also reported receiving information from radio and a little over 20% from strangers, family and friends via text messages, phone calls, face-to-face conversations and new social media such as facebook and twitter.

 Athena Kolbe is a researcher at the University of Michigan. Dr. Robert Muggah is Research Director at the Small Arms Survey and Visiting Professor at the Graduate Institute of International and Development Studies, Geneva.

[1] A. Kolbe and R. Muggah, Surveying Needs After the Quake: Results of a Randomized Household Survey in Haiti (New York/Geneva: UNDP and IDRC, 2010).